MOMENT CONDITIONS AND SUPPORT THEOREMS FOR RADON 
TRANSFORMS ON AFFINE GRASSMANN MANIFOLDS 



FULTON B. GONZALEZ AND TOMOYUKI KAKEHI 



Abstract. Let G(p, n) and G(q, n) be the affine Grassmann manifolds of p- and q- planes in 
R™, respectively, and let TZiP'l) be the Radon transform from smooth functions on G(p,n) to 
smooth functions on G(q, n) arising from the inclusion incidence relation. When p < q and 
dim G(p,n) = dim G{p,n), we present a range characterization theorem for "R,(p>i) via moment 
conditions. We then use this range result to prove a support theorem for 1Z (P'l) . This complements 
a previous range characterization theorem for 1Z ' p ' 9 ' via differential equations when dim G(p, n) < 
dim G(p,n). We also present a support theorem in this latter case. 

1. Introduction. 

In this paper, we present a range characterization of Radon transforms on affine Grassmann 
manifolds via moment conditions. Our objective is to generalize the moment conditions for the 
classical Radon transform on R n to affine Grassmannians; our results complement a previously 
obtained range characterization for these transforms using invariant differential equations. 

Specifically, we consider the transform TZ <yP,q ^ from smooth functions on the space G(p, n) of p- 
planes in WL n to smooth functions on the space G(q, n) of g-planes in M." arising from the inclusion 
incidence relation. Throughout this paper, we assume that p < q. A p-plane I and a g-plane £ are 
incident if £ C £; the Radon transform is explicitly defined by 

(l.i) nMfit) = f f(£)de£. 



when £ 6 G(q,n) and / is a function on G(p,n). Here d^£ is a canonical measure on the set 
£ = { £ G G{p, n) \£ is incident to £ } invariant under all Euclidean motions preserving r\. In 
particular, when p = 0, TZ^ '^ reduces to the classical g-plane transform on R™. We will be 
working in the category S(G(p, n)) of Schwartz class functions on G(p,n), as defined in |Ri| . The 
following theorem [GK 1, Theorem 7.7] gives a range characterization of TZ^ p ' q ^ in the case when 
dimG(p, 7i ) < dim G(q,n): 

Theorem 1.1. Assume that p < q and dimG(p, n) < dmiG(q,n). Then there exists a differential 
operator of order 2p + 4, invariant under the Euclidean motion group M(n), such that for any 
<peS(G(q,n)), 

(1.2) V £ K M S{G(p,n)) 0^ = 0. 

This theorem generalizes the range characterization of the fc-plane transform on R n , when k < 
n — 1, via a second order ultrahyperbolic system (PS], |Go3j ) a single 4th order M(n)-invariant 
differential operator f |Go2| h 
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The classical Radon transform, on the other hand, has its range given by moment conditions, as 
specified in |GGV| . |Lu| . |H1| . (See the last reference for a complete proof.) The Grassmannian 
analogue to this would correspond to the transform 1Z ^ p,q ^ with dim G(p, n) = dim G(q, n) . Since the 
dimensions coincide, we would expect the range to be specified by appropriate moment conditions 
as well. This is the content of our main result, Theorem 13.11 below. As with the proof in |H1| . 
the crucial clement of our proof consists of justifying the smoothness of a certain "partial Fourier 
transform" on the space G(p, n). (Propositions 13 . ll and 13 . 2l below. ) 

Now Helgason's geometric proof of the support theorem for the classical Radon transform is 
well-known ( |H1| : less well-known is the fact that it is also a consequence of the forward (easy) 
moment conditions and a polar-coordinate version of the Paley- Wiener Theorem ( H3 ; Theorem 12. 31 
below)). We give an extension of this theorem to affine Grassmannians ( Theorem 16. 1|) and use it to 
prove a support theorem ( Theorem 16.2(1 for the transform TZ^^ in the case when dim(G(p, n)) = 
dim(G(q,n)). The injectivity of TZ^ p,q ^ is then used to prove the support theorem fTheorem 17.11) 
when dim(G(p, n)) < dim(G(g, n)). 

2. Moment Conditions and the Support Theorem for the Classical Radon 

Transform, Revisited. 

To clarify our exposition, it will be instructive to briefly summarize Helgason's proof of the range 
characterization of the classical Radon transform by moment conditions. For this, let 7Z = : 
/ i— > / denote the classical Radon transform, which maps functions in 5(R") to functions ip G 
^(S™- 1 x R) which satisfy (p(u,p) = ip(-u, -p). (See |H3], pg.99 for the definition of ^(S™" 1 x R).) 
We call such functions even and define Sh (S" _1 x R) to be the vector space of all even functions in 
S(S n ^ 1 x R) satisfying the following condition: 

For any k G Z + , there exists a homogeneous polynomial Pk of degree k on R™ such that 

(H) f°° 

/ <p(uj,p)p k dp = P fe (w), forueS"- 1 . 

Theorem 2.1. (^) nS(R n ) = ^(S"" 1 x R). 
It is an easy calculation to show that 

/ nj{LO i p)p k dp = f f( X )(x,L0) k dx. 
J -oo JR n 

This shows that TZS(W l ) C S^S"" 1 x R). Conversely, suppose that ip G ^(S"" 1 x R). Define 
the "partial Fourier transform" ip on S™ _1 x R, 

/oo 
(p(uj, P ) e~ ips dp, set, we s n_1 . 
-oo 

It is straightforward to prove that tp is an even function in 5(S™ -1 x R). In addition, the condition 
(pT|) for k = shows that ui i— ► <p(u}, 0) is constant so that there exists a unique function F on R™ 
for which F(sui) = (p(u>, s). Now the map (ui, s) i— > sw is a local diffeomorphism of S" _1 x (R \ {0}) 
onto R ra \ {0}, which shows that F is smooth outside the origin. To prove the smoothness of F at 
the origin, it suffices to show that the partial derivatives of F are bounded on, say, the punctured 
unit ball B' = { x G R" | < ||a;|| < 1 }. Fix e > 0. Now on the subset A n<e = {suj G B' \ < 
s < 1, co = (uji, ■ ■ ■ ,uj n ) G S n_1 , u> n > e > 0} we can use (s,wi, • • • , w n _i) as local coordinates: 
repeated application of the chain rule shows that 

(2.2) - dk * {x) = J2 A ^-^;-^-\ d \ * F{au) 

y ' dx u ■ ■ ■ dx lh y ' ^ s k -i du u ■■■doji ds^ y ' 
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The coefficients Aj-i lt ... t i m (wi, • • • , w n _i) are smooth bounded functions of (uj\, • • • , oj n ~i) and the 
right hand sum ranges over all j < k and over all sequences ii, ■ ■ • ,i m in {1> 2, • ■ • , n — 1} where 
m < k. 

We write e~ lps = Yli=o(~^P s ) 1 + e k{~ips) and apply condition JHJ to (|2.1|) to obtain 
F(sw) = ^ — — Pi(suj) + / ip(u),p)e k (-ips)dp. 
Since the Pj are polynomials of degree I, (|2.2[1 implies that 

Now G x K) and ek- {—it) / '{—it)^ 3 is bounded for all real t, so (|2.3|) shows that 

the kth order derivatives of F are bounded on the set A n ^ e . From this, we deduce that the fcth 
order derivatives of F are bounded on B' . Hence F £ C°°(R n ). A routine calculation using, 
say, l]2.2[l . shows that F £ S(W n ). Denoting the Fourier transform on IR n by J 7 , let / be the 
inverse Fourier transform of F: F = J-(f). For any ui, s the projection-slice theorem says that 
J-f(su) — J_ f(Lu,p)e~ tps dp. The injectivity of the Fourier transform on K then implies that 
<p = f.U 

We now state the support theorem for the classical Radon transform in the following form. 

Theorem 2.2. flHH ) Let f e S(R n )_ and suppose that R > 0. If f(u,p) = whenever \p\ > R, 
then f has support in the closed ball Br — {x G R" | ||x|| < R}. 

While the support theorem can be proved geometrically, it is also a consequence of the forward 
moment conditions (H). The key is the following polar coordinate version of the Paley- Wiener 
theorem. (See exercise Bl, Ch. 1 in |H3| . 1 1 



Theorem 2.3. (H2 ) Let R > 0. The Fourier transform f i— > Tf maps V(Br) onto the set of 
functions J-f(Xuj) — ip(u), A) G C oc (S n ^ 1 x R) satisfying the following conditions: 
(i) For each uj G S n ~ 1 , the function A i— > ip(u>, A) extends to a holomorphic function on C with the 
property that 



sup 

u?, A 



^( W ,A)(1 + |A|) ; 



< oo N G Z 4 



(ii) For each k G Z + and each homogeneous degree k spherical harmonic function h on S n 1 , the 
function 



Ah) ' ip(u,X)h(u)duj 

is even and holomorphic on C (du> denoting area measure on S 1 "" 1 ). 

To see how the support theorem follows from this, we take / G <S(R n ) satisfying f(u;,p) = for 
all \p\ > R. Then ip(ui, s) — J 7 f(suj) satisfies condition (i) above by the easy part of the Paley- 
Wiener theorem and the projection-slice theorem. Now let h be a homogeneous degree k spherical 



The authors would like to thank Prof. S. Helgason for pointing out this exercise. 
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harmonic on S n . Then 



S"" 1 J-co 



ip(uj, s) h(u)) du> = I / f(u>,p)e tps dp h(us) dus 

iS™- 1 J-oo 
fe — 1 ^ /> oo 

J 1 J Mp s )' dph(uj) doj 

f(uj,p) h{uo) duo e k (-ips) dp 

/oo 
f(u,p) h(u) dui ek(-ips) dp 
-oo 

by the forward moment conditions (H) for /. Since Pi{oj) is a sum of spherical harmonics of degree 
< I, the sum on the right-hand side vanishes, so that 

s~ k / ip(uj, s) h(ui) dej = / s~ k ek(~ips) / f{oj,p)h{uj)dujdp. 

JS"- 1 J-oo JS"- 1 

The inner integral on the right is a smooth compactly supported function of p G R and s ^ 
s~ k ek(—ips) extends to a holomorphic function on C. In addition, the right hand side above is 
easily seen to be even in s. Hence ip(u>, s) satisfies condition (ii) in Theorem 12.31 and so / is 
supported in the closed ball Br. 



3. The range of the Radon transform on affine Grassmannians: the equal rank 

CASE 

We adopt the notation of |GK1| in what follows. Let G p _ n be the (compact) Grassmann manifold 
of p-dimensional subspaces of R™. Then G p . n = 0(n)/K p , where K p = 0(p) x 0(n — p) is the 
subgroup of 0(n) fixing the p-plane <jq = Rei + • • • Re p . 

We assume that the Haar measure on all compact Lie groups, and the invariant measures on 
their homogeneous spaces (with the exception of the unit spheres), are normalized. Let R P , q , n ■ 
C°°(G P)n ) — » G°°(G q<n ) be the Radon transform corresponding to the inclusion incidence re- 
lation between p- and q - dimensional subspaces of R™. Then R p , q , n is a linear bijection when 
rank(G P! „) = rank(G ?i „) ( ( I r ) ; when rank(G Pi „) < rank(G g ,„), R P , q , n is injective and the range 
Rp,q,n G°°(Gp in ) is the subspace of G°°(G gjn ) annihilated by an 0(n)-invariant differential operator 
of order 2rank(G p , n ) + 2. ffKj. [GKT^ 

When rank(G p ,„) < rank(G 9 , n ), there is an 0(n)-invariant operator D Pt q jn : G°°(G Pifl ) — > 
G°°(Gp in ) which inverts -Rp, g , n : 

(3.1) / = Dp^.n R q . p .n Rp,q,n /; / G G (Gp >n ). 

The operator D p ^ n , given explicitly in [Ip, corresponds to multiplication by a constant factor 
on each of the if-types in L 2 (Gp jn ), and is a differential operator when q — p is even. We call O p , q , n 
a reproducing operator. 

As stated in the introduction, we assume that p < q. Let 770 denote the g-plane Rei + • • • + Re g , 
and let H p and H q denote the subgroups of the Euclidean motion group M(n) = 0(n) x R™ fixing 
the p-plane oo and the g-plane 770, respectively. We have, in particular, H p = (0{p) x 0(n—p)) x R p 
andG(p,n) = M(n)/H p . 
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For suitable compatible measures on M(n), H p , H q , and H p n H q , the affine Grassmannian 
transform TZ^ p ' q ^ is the Radon transform associated with the double fibration 

M{n)/(H p nH q ) 

s \ 

G(p, n) = M(n)/H p G{q, n) = M{n)/H q 

The corresponding incidence relation between p- and q -planes in l n is just inclusion. 

We can write TZ^ p,q ^ explicitly in the following way ( |(tK1| ). G(p, n) is a vector bundle over G p .n 
of rank n — p: its fiber over each a G G p . n is ; each I G G(p, n) is written uniquely as I = (cr, cc) , 
where cr is the parallel translate of I through the origin and {x} = a 1 - n t. We parametrize G(q, n) 
in a similar manner: G(q,n) — {(t],v) G G g ,„ x R" 1 1> _!_ 77}. From |GK1| the transform IZ^^ is 
then given by the formula 

(3.2) = / (/ /(cr.w + aOdaA da, 

•/(XCjj \Jo-- L n7) / 

for any appropriate function / on G(p, n). The outer integral is taken over the set {a G G Pin \ a C 77}, 
with respect to the normalized measure invariant under all u G 0(n) preserving 77. 

Let S(G(p,n)) and S(G(q,n)) denote the spaces of Schwartz-class functions on G(p.n) and 
G{q,n), respectively. Then : S{G(p,n)) -> S{G{q,n)), by [(TkT) . §6. Next let T p and JF, 

denote the partial Fourier transform (i.e. Fourier transform on the fibers) on G(p, n) and G(q,n), 
respectively: 

(3.3) ?pf{?,v) = I f(a,x)e- l ^dx, feS(G(p,n)). 

Using (|3.2|l , it is not hard to prove the affine Grassmannian version of the projection-slice theorem: 

(3.4) F q °rt M) f{n,y) = / Fpf{o-,y)da 

for all (r?,y) G G(q,n). f jcTRT] . Prop. 6.1.) 

We define the rank of G(p, n) to be min(p + l,n — p). Since dim G(p, n) = (p + l)(n — p), we 
note that rank(G(p, n)) < rank(G(q, n)) if and only if dimG(p, n) < dim G(q,n). The range result 
Theorem 11.11 from |GK1| is essentially proved using the projection-slice theorem and the range 
characterization of the compact Radon transform R Piq on Grassmannians in R™ _1 via 0(n — 1)- 
invariant differential operators. In addition, the projection-slice theorem and the inversion formula 
(|3.1(l for R Ptq on R n_1 are used to prove an inversion formula (Theorem 6.4 in |GKlj ) for 7^ p,<? ' 
when dim G(p,n) = dim G(q,n) and q — p is even. (See Rubin |Ru2j for another inversion formula 
for TZ^P'^ without the parity restriction.) 

It is not hard to obtain an analogue of the moment conditions (H) for the transform TZ( p ' q \ Let 
k G Z + and / G S(G(p,n)). Then for any (77, y) G G(q,n) we have 

/ n^f( V ,v)(v,y) k dv 

(3.5) 



/ (v, y) I / /(a, v + x) dx da dv 



/(a, v + x) {v + x, y) dx dv da 



<J<Zr\ J a 



/(a, w) (w, y) dw da. 
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Here we have used the fact that a equals the orthogonal direct sum (er n 77) (Br]- 1 . The inner 
integral above represents a smooth function Pk(a,y) on G(p,n): 

(3.6) Pk{<J,y) = j f(<r,w) {w,y) k dw, y€a x . 

Clearly P k is a homogeneous degree k polynomial on the fibers of G(p, n). 

Equations (|3.5() and l|3.6|l lead us to define SH{G(q,n)) as the vector space consisting of all 
ip G S(G(q, n)) satisfying the following condition 

(if')' For each k G Z + , there exists a C°° function Pk on G(p,n) such that 

(1) For any a G G pn , the function y i— > Pk(a,y) is a homogeneous polynomial of degree k on 

(2) For all (f],y) G G(q,n), we have 

(3.7) / tp(r),v) {v,y) k dv = / P k {a,y)da. 

J r/ 1 - J <T<Z.r\ 

(|3.5|) shows that the range is a subspace of Su(G(q,n)). Note that the condition 

(H') reduces to the classical condition (H) when p = and q = n — 1. We now state our main 
result: 

Theorem 3.1. Suppose that p < q and rank(G(p, n)) = rank(G(q, n)). Then TZ^ p ' q ^ S(G{p,n)) = 
S H {G{q,n)). 

Our proof roughly follows the lines of the classical proof. Let (p G S}i(G(q,n)). The partial 
Fourier transform ip of tp, 

(3.8) tp(r),y) = T q <p(w) = / v(»7> «) e" 1 ^ dw, 

belongs to S(G(q,n)) by jcTRTj . We now introduce the "flag" manifold F q , n = {(i], us) G G gi „ x 
gn-i | ^ j_ w | (Define F PtU similarly.) Then <p gives rise to a smooth function $ on .F 9jn x R 

(3.9) $(?7,w,r) = (p(rj,ru)). 

Note that $(77, w, r) = $(77, — w, — r). 

For each u G S*™ -1 , let G p (oj ) and G^(w J -) denote the compact Grassmann manifolds of p- and 
g-dimensional subspaces of the (n — l)-dimensional space ui 1 - C M". Then G p (o; ) and G q {uj x ) 
are diffeomorphic to G Pi „_i and G g ,„_i, respectively, and are homogeneous spaces of the subgroup 
0(ui) of O(n) fixing to. 

F q _ n is a fiber bundle over S 1 ™ -1 with fibers G p (w x ). If we identify the Grassmannian G q>n -i with 
G g (e^), we can see that F q ^ n is the associated fiber bundle 0(n) X-o(n-i) G q ,n-i of the principal 
bundle 0(n) — » 5 n_1 , u 1— > u • e„. Let 7r 9 : O(n) x G q>n -\ — » i^, n j (w, 7 ?) I— * (u- -q,u- e n ) be the 
quotient map. Using local cross sections, it is easy to see that a function $ is smooth on F qn iff its 
lift $ o n q is smooth on 0(n) x G q , n -i- 

For each w G S n -\ let fl£ g : C°°(G p (w x )) -> G°°(G g (w x )) be the 0(w)-invariant Radon trans- 
form corresponding to the inclusion incidence relation, and let R qp be the dual transform. R£ is of 
course just a translate, under O(ra), of the transform i? Pi9jn _i : G°°(Gp in _i) — ► G°°(Gg i „_i) defined 
in the beginning of this section. Since p + q+ 1 = ri, it follows that rank(G p (w ± )) = rank(Gg(u> )) = 
min(p, g), and so R p q is a linear bijection. Let be the corresponding reproducing operator; the 
appropriate translate of jSUJ for i?^ is 

(3.10) ^ = Ul q Rl p oR%^, 
for all V G G 00 (Gp(w- L )). 
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Let us now return to the function $ on F q . n x R. Since R p q is a bijection, there is, for each 
(u>,r), a unique smooth function F(-,uj;r) on G p such that 



(3.11) *fa,w;r) = / F(a,co;r)da, 

for all (77, u;r) G F Pi „ x R. We also express this as r) = ^R pq F(-, w; r)J (77). We can, of 

course, think of F as being a function on F p>n x R. 

We want to prove that F is smooth on F Pi „ x R. (This is not completely obvious.) Since the 
variable r is fixed in i|3.11fl . this reduces to showing that the map (ct, ui) 1— > F(<j 7 uj; r) is G°° on F Pi „ 
for each r. In view of the inversion formula 13.10fl . let us consider the integral transform 5", from 
functions on F q ^ n to functions on F Pj „, given by 

(3.12) S$(a,u) = f *(ri,w)dr) = (E%j,*(;w))(a), (<r,«) G F p>n 

for all <E> G C°°{Fq tn ). Here dij denotes the canonical and normalized measure on the submanifold 
{V G Gg,„ I a C 77 C w x } of G 9: „. 

Lemma 3.1. S is a continuous linear operator from C 00 {F qtTl ) to C°°(F p ^ n ). 
Proof. In fact S is the Radon transform associated with the double fibration 

0(n)/(K q r\K p ) 

S ^ 

F q , n = 0(n)/K q F p>n = 0(n)/K p 

where K q = 0(q) x 0(n — q — 1) and K p = 0(p) x 0(n — p — 1) are the subgroups of 0(n) 
fixing (i]Q,e n ) G F q ^ n and (ao,e n ) G F Pj „, respectively. Hence by Ch. I, Proposition 3.8 of |H3| . the 
transform 5 is a continuous linear operator from C°°(F q:n ) to G°°(F P! „). □ 

It will be useful to express S in terms of associated fiber bundles. In terms of the quotient maps 
iT q : 0(n) x G q , n -i -> F 9: „ and tt p : 0(n) x G p ,„_i — > F p ,„, we have 

(3.13) S<S>on p (u,a) = J $ o % q (u, T}) drj = (R q , Ptn -i($ o 7c q )(u, ■)) (cr). 

If $ is smooth on F g! „, then the right hand side is smooth on 0(n) x G p ,„_i. (This also shows that 
5$ is smooth on F p>n whenever $ is smooth in F q ^ n .) 
Next we define the operator on F p .„ by putting 

(3.14) nWF( ff , u ) = w )(<7) 

for all F G G°°(F p .„). (In the above, the operator acts on the first argument.) 
Lemma 3.2. is a continuous linear operator on G°°(F Pjn ). 

Proof. Again letting tt p : 0(n) x G p , n -i — > F Pjn be the quotient map, the 0(n — l)-invariance of 
O p . q . n -i implies that 

(3.15) (n^G)on p (u,a) = n p<g<n ^(Go^ p )(u,a), for G G G°°(F Pi „) 

where Dp^.n-i acts on the second argument. Therefore, the continuity of follows from (|3.15|) 
and the continuity of Dp,,, □ 
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In particular, D^F is a smooth function on F p ^ n . 

We now slightly modify the definitions of the operators S and in Ij3.12|l and (|3.14|) so that 
they act on functions on F qxfl x M and F p ^ n x R, respectively. In other words we put 

(3.16) S$(a,u;r) = f lu; r) drj, (a,uj)eF Ptn 
for all # G C°°(Fg,„ x R), and 

(3.17) □<*V( £ r,w;r) = C£ 9 y(.,w;r)(<r) 

for all V G C°°(F p n x R). Lemmas 13.11 and 13.21 suitably modified, still apply to show that S : 
C°°(F qtn x R) -> (7°°(Fp, n x R) and : C°°(F p , n X R) -> C°°(F Ptn x R) are continuous linear 
operators. 

Now by the inversion formula l|3.10|l for i?^ and definitions l|3.12|l and l|3.14[l . we can recover F 
from $ in equation Q3.11|l : 

(3.18) F(a,Lj;r) = (d (p) o S 1 "!) ) (er, tj; r). 

By the remarks above, we see that F € C°°(F P}n x R). 

The uniqueness of F 1 in H3.11|l implies that F(cr, u>; r) — F(a, —lu; — r) for all (a, lu; r) £ F p . n x R. 
We next show that the moment conditions (H 1 ) for k = imply that -F(<7, w;0) is constant in 
a; G 5 n_1 (la 1 . The function Pq((J, v) on G(p, n) given in (H 1 ) in this case is a Oth degree polynomial 
in v for each a G G Pi „, and thus depends only on <r, so we put Pq((t) — Pq(<7,0). We have 
P^ G G°°(G p ,„) and by E3, 

If we take any lu G and 77 G G^ , we have by l|3.9|) and l|3.11jl . 

0(77,0) = a;, 0) = / F(a,LO,0)da. 

J uCr\ 

Since R p q is injective we see that F(cr, u>,0) = Po(cr) for all a _L u>. 

In view of this and the fact that F is even in (lu, r), there exists a function / on G(p, 71) given by 

(3.19) f(a,rcu) = F{a,cu;r), (a,w; r) G F p ,„ x R. 

Now the mapping (a, lu; r) 1— > (<j, rw) is a local diffeomorphism from F PtU x (R\{0}) onto G(p, n)\G p>n . 
(This is best seen by viewing both F Pi „ x R and G(p, n) as bundles over G Pi „, or as associated bundles 
of the principal bundle O(n) — > G P) „, u 1— ► u • <ro-) 

/ is therefore smooth on G(p,n) \ G Pjra . In addition, it is continuous on G(p,n), since the map 
(<j,lu; r) f— > (cr, rw) is a quotient map of -F Pi „ x R onto G(p, 71). From equation l|3.11|l . / satisfies the 
relation 



(3.20) 0(7,, y) = / f(a,y)da 

J crCr; 

for all (77, y) G G(q, n). 

Our next objective, given in Propositions 13 . II and 13 . 21 below, is to prove that / is smooth on all of 
G(p, n), and that in fact / G S(G(p, n). Assuming this, let / be the inverse partial Fourier transform 



MOMENT CONDITIONS AND SUPPORT THEOREMS 



<) 



of /: T v f = f. Then the projection-slice theorem Q£ 41 in conjunction with equations jfl.lip . I|H.8|I 
$nfy . and (pH5)) show that 

for all (j], u>) £ F q ^ n and all r £ K. By the injectivity of JF 9 , we get TZ^ p ' q ^ f = y>, which proves 
Theorem O 

Proposition 3.1. f € C°°(G(p,ri)). 
Proposition 3.2. / £ S(G(p,n)). 

We will give the proofs of the above two propositions in Sectional 

4. Differential operators on Grassmann manifolds and flag manifolds 

In this section, we will study the calculus of differential operators on F PiU and on G{p, n) \ (G Ptn x 
{0}) = F p<n x R + . (Here we identify G Pj „(c G(p,n)) with G Pi „ x {0}, using the parametrization 
G(p,n) 3 £ = (<t,x), a £ G Ptn ,x £ a .) In particular, we will give a kind of polar coordinate 
decomposition of differential operators on G(p,n) \ (G p , n x {0}) analogous to l|2.2(l . The results in 
this section will be applied to the proofs of Propositions 13 . II and \'3 . 21 in Section [S] 

Let M(n) — 0(n) x R™ be the Euclidean motion group, and let m(n) = so(n) x M." be its Lie 
algebra. m(n) has basis consisting of the elementary skew symmetric matrices Xij — Eij — Eji (1 < 
i < j < n) and Ek (1 < k < n), where E\,--- ,E n denote the infinitesimal translations in the 
directions of the unit vectors ei, ■ ■ • ,e„. Let u(m(n)) denote the universal enveloping algebra of 
m(n). We note that u(R") C u(m(n)) is a commutative subalgebra of u(m(n)). We call P € u(R") a 
homogeneous element if P is written as a homogeneous polynomial of Ei, ■ ■ ■ , E n . By the Poincare- 
Birkhoff-Witt theorem, the following proposition is easily obtained. 

Proposition 4.1. Any element U £ u(m(n)) is written as a linear combination of terms of the 
form QP, where Q £ u(so(n)) and P £ u(R ra ) is a homogeneous element. 

For the sake of simplicity, throughout this section, we write the action of X £ m(n) on / £ 
C°°(G(p,n) as 

X ■ M* t x)) = ^/(exp(-tX) • (a,x))\ t=0 . 

Similarly, if X £ so(n) and / £ C°°(F p _ n ) (or / £ C 00 (G PiIl ) ), we write the action of X on / as 
X ■ f. In addition, we also write the action of U £ u(m(n)) on / £ C°°(G(p, n) as U ■ f. 
For v £ S n_1 and a > 0, we introduce the open sets 

Fp,n( v ) = {(o-,u)eF p , n \\\Pr a xv\\>a}, 

Pr a ± denoting orthogonal projection to a . In addition, for an index set / = { i\ 1 ■ ■ ■ ,i m } with 
1 < ii < ■ ■ ■ < i m < n, let 

m 
fc=l 

Then we have the following. 
Lemma 4.1. 

U F p Q n[ J ] = F p.™> for some a > 0. 
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Proof. Take an arbitrary point (a, u>) G F pjl . Since a is a p-dimensional subspace of R™, there 



exist n — p unit vectors , 
So if we put 



(1 < ii < • ■ • < i n - P < n) such that ei k £ a, (1 < k < n — p). 



a = - min{ ||Pr CT _L e ifc | | | 1 < fc < n - p }, J = { i lr •• ,i„_ p }, 

then (<7,a>) G ^ n [^]- Let 14 = U#/= n -p ^"nW- Then, the above result shows that U Q >o V<* = 
-Fp.„. Since { } Q >o is an open covering of the compact set F Pt1l , we can take a finite covering 
{ V aj }jLi of F PtTl . Let ao = mini<j<Ar ctj. Then we have V ao — F p , n , which proves the assertion. □ 

Lemma 4.2. Let Un = { = {0\,--- ,6n) G S^ -1 \ On > }■ There exist smooth functions 
dj (j = 1, • • • , N — 1) and 6 on S w_1 x Wat smc/i that 



N-l 



(E v f)(r9) = ]T 



?2^X jN f{rO)+b{v,Q)^f(rO), 



/or (u, 6») G S 



iV-l 



Wat, r>0, and /or / G C°°(R \ {0}). 



Here E v denotes the directional derivative in the direction of —v G K , namely, 



E v ■ f{x) = ^f( x - tv )\t=Q, 



Proof. It is not hard to see that 



forfeC c 



-XjN 

r 



d 



'N 



dx-j 



d 



8xn' dr 



N 



d 



3 = 1 J 



Hence 



Note that 



/ \Xi N 



d 

dr ' 



(8n 
On 



(0 n 

N 

••• 

\0i 2 



un 

&N-1 



det 



••• 
\0i 2 



9i \ 



On —Vn-i 
On-i On ) 



-62 



-vn-i 
On J 



Q N-2 
7 N 



Thus, each vector field is written in the form, 



/ -2- \ 



a 



dXN-l 

V dx N / 



>0, if 6>G Un- 



d %^ a ok {0) 1 6,(0) d 



9x 



k=l °N 



N ~ 2 dr' 



where ajk(0) and &j(0) are polynomials of G Wat. Since E v = — J2jLi v . 
from the above expression of 



■> l)x 



— , the assertion follows 



□ 
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Next, in a similar way to the case of we introduce the radial derivative E r on G(p, n) \ (G Pi n x 
{0}) and the directional derivative E v on G(p,n) in the direction of — v 6 R" as follows. 

E r f(a,x) = J-/(a,rx)| r=1 , for / e G°°(G(p,n) \ (G p ,„ x {0})), 

K ■ f(£) = j t f(t - tv)\ t=0 , for / G G°°(G(p,n) \ (G p ,„ x {0})). 
Our aim now is to generalize 12.211 to G(p, n). 

Proposition 4.2. For an arbitrary point (o~i,lui) G F Ptn , there exist an open neighborhood U of 
(dijCiJi) in Fp. n and smooth functions a\j(o-,uj), b l (o-,uj) on U such that 

(4.1) E l -f{a,ru)= £ ^(a, w) X ti ■ f{a,ru) + b l (a,u) E r f{a,ru), 

l<«j'<n 

forl(l<l< n), (cr,w) £U,r>0 and for f e C°° (G(p,n)\G Pt „ x {0}). 

Proof. Let us first take an open neighborhood Vi of &\ in G p%n and a smooth local cross section 
u : Vi — > SO(n) such that 

u(cr)<7 = Mei © • • ■ © Re p ( we put = co); for <r G Vi, u(<7i)a>i = e n . 

Then, by taking a sufficiently small open neighborhood £~2i of lo\ in S™ _1 , we have 

u(a-)w G { (0, • • ■ , 0, 0i, • • • , 0„- P ) G S"- 1 | 6-„_ p > }, 

for (a,u>) G (Vi x fi x ) n F Pj „. Now, let 

w = (Vi x fii)nv 

By Lemma f4.ll there exists an index set I = { ii, ■ ■ ■ , i n - p } such that W C F^ n [I\. (Replace Vi 
and Oi by smaller neighborhoods if necessary.) 

Step 1. Let us first consider the case I G I. 

The vector e; can be written as 

e ; = Pr CT e/ + Ri(a) yi(a), 
(4 ' 2) = 1^x6,11, Vl (a)= Z Ta±e 'n eS"- 1 ^. 

By the definition of F£ n [I], 

Ri(a) = \\Pr a ±ei\\ > a, for a G Vi. 

Therefore, Ri(a) and ?//(cr) are smooth on Vi. Moreover, by l|4.2|l 

• /((7,rw) = Ri(o-)E yi(a) ■ f(a,rcu), for / G C°° (G(p,n) \ G p , n x {0}). 

From now on, we will decompose the above vector field E yi t a \ as a linear combination of rotational 
derivatives and the radial derivative E r . Since u{a) ■ yi{cr) G o-q = Re p+ i © ■ ■ • © M.e n = M. n ~ p , we 
can apply Lemma l4.2l to the case when N = n — p and v = u(cr) ■ yi{o). Thus 

n — 1 ^ 

, s ^u(a).y,(<7) -flM) = J! -AnW^'S/iWi^^mn-ff)^) 

V 4 -^ m= P +l 

+ B(u(a)- yi (a),8)(E r g)(r9), 

for 6> = (0, • ■ ■ ,0,6»i, ■ • ■ ,9 n - p ) G cr^ n S"" 1 , 6>„_ p > 0, r > 0, and for g G G°°(JR™- p \ {0}). Here 
A m and B are smooth functions on S"^- 1 x { 6 = (0, • • • , 0, 6 X , ■ ■ ■ , 8 n - p ) G cr^nS"- 1 | 6>„_ p > }. 
(We identify 6 with a unit vector (6>i, • • • , 9 n - P ) in W l - p .) 
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Let us take g(r9) = /(li(cr)" 1 • a , ru(o-)- 1 • 9) in g^J. Then, 

E V,(a) ■ /(o-)TO) = E u{a) . yi(a) ■ /(u(cr) _1 • cr , ru(cr)" 1 • 0)| e=u(ff) . w 
n-1 j 

= ^2 - A m{u{<j) ■ yi{a),9) X mn ■ J(u(<t) 1 • a , ru(a) 1 



|0=u(<t)-w 



m— p+1 

(4.4) +B(u(o-)-yi(^)^)-Er/K<T) _1 -^o, nt(o-)- 1 • e)\ e = u{a y u 

n—1 1 

= E -A m (u(a)- m (a),9)(X^-f)(a,rcj) 

+ B(u(<7) • W (<7),fl) /)(a,ru,). 

We note that the radial derivative E r is invariant under the action of SO(n), in particular, Er^ a ^ = 
E r . In addition, we also note that J^n is written in the form 

(4-5) J2 

l<i<j<n 

where C™(<r) is a smooth function on V%. Combining (|4.4|l and l|4.5[) . we have the following expres- 
sion 

(4.6) E yi{a) ■ f(a,ruj) = - a ^(a, w) Xij ■ f(a, ru) + b\a, u) E r f(a, rui), 

l<z<j<n 

for some smooth functions a\j and b on U. Since Ri(cr) is smooth on Vi, we obtain an expression 
of the form (|4. 1|) for I € J. 

Step 2. Next, let us consider the case I ^ /. Since U C Fp n [T\, we see easily that 

(4.7) R" = g © (e n , ■ • • ,e,„_ p ), for a G Vi, 

where (e^ , • ■ • , &% n _ } denotes the (n — p)-dimensional subspace spanned by , • ■ • , ei n _ . (Note 
that the decomposition l|4.7|) is not necessarily an orthogonal decomposition.) Using the decompo- 
sition (|4.7() . we can write e; in the form 

n—p 

e; = z + ^ a fc (tr)e lfc , z e a. 
fe=i 

Obviously the coefficient a,k(a) is a smooth function of a 6 Vi. Thus, 

n—p 

(4.8) Ei-f{a,rw)=Y d 

k=l 

From l|4.8|l and the result of Step 1, we can conclude that Eif(a, ruo) — E ei f(a, ru) has an expression 
of the form (f^TTf) for I ^ I. □ 

Proposition 14.21 yields the following. 

Proposition 4.3. There exist smooth functions a\j(o-,ui) and b l (a,uj) on F p ^ n such that 

(4.9) Ei-f(a,ru)= £ ^(<7,w)X« • f(a,ru) +b l (a,w) E r f(a,ruj), 

l<i<j<n 

forl(l<l< n), (a, to) e F p . n , r > and for f € C°° (G(p, n) \ G p>n x {0}). 
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Proof. Since F PiTl is compact, it follows from Proposition ^. 21 that there exists a finite open covering 
{Uv}i<i/<m °f Fp.n such that Eif has an expression of the form l|4.1J) on each U u . So the above 
(globally defined) smooth functions a\j(a,oj) and b l (o~,uj) can be constructed, using a partition of 
unity for the open covering {U u }i< u < m . □ 

Let us consider u(K™) to be a subalgebra of u(m(n)) = u(so(n) © R"). We see easily that an 
element of u(R") is written as a polynomial of vector fields Ei, ■ ■ ■ , E n . The following theorem is 
the affine Grassmann version of equation l|2.2[) . 

Theorem 4.1. Let P = P( m '(Ei, • • • , E n ) 6 u(R") be a homogeneous element of order m. Then 
for f G C°°(G{p, n) \ G p . n x {0}), P {m \E 1 , •••,£„)• f(a, ru;) is expressed as 

m 

P< m >(£i,... ,E n ).f(*,ru>) = J2 Ai m) (D (a ^)r- k El n - k f(a,ru>), for (a,u>) e F p , n , r > 0, 



fc=0 



where A ™ (D( a ^) is a differential operator on F p ^ n of order at most k. 

Proof. We will prove the theorem by induction with respect to m. In the case m = 1, the assertion 
follows from Proposition 14.31 Suppose that the assertion of the theorem holds for any element of 
u(R") of order at most m. Let us take any homogeneous element Q £ u(R") of order m+1. Then, Q 
can be written as a linear combination of EjQ^ (1 < j < n), where Qw 6 u(R n ) is a homogeneous 
element of order m. Therefore, without loss of generality we may assume that Q = E\P for some 
homogeneous element P 6 u(K") of order m. Then, by the hypothesis of induction, Pf(o-,rui) is 
written as 

m 

P(E U --- ,E n )f(a,ruj) = £ A k (D {a ^))r~ k E™~ k f(a, ruj) for {a,u) e F p , n , r > 0, 

k=0 

where Ak(Dt a>UJ )) is a differential operator on F p>n of order at most k. 

On the other hand, by Proposition ^. 31 there exist smooth functions aij(o~,u>) and &(<r, to) on 
such that 

Ei ■ F(a,ruj) = -aij{a,uj) X^ ■ F(o-,ruj) + b(a,ui) E r F(a,ru>), 

l<i<j<n 

for (<r, uj) S F p „, r > and for F £ C°° (G(p, n) \ G Pi „ x {0}). Here we note that 

rXij = lyr, E r Xij = X i3 -£V, rA*(.D (ffiU )) = A fe (£) (CT)W) )r, E r A k (D {rT>ul) ) = A k {D {rT>ul) )E r . 
Thus we have 

Q-f{c-,ru>)=E 1 P-f{o-,ru) 

m 

= E T, a ^^ X ^MD { ^ ) )r- k - 1 E^- k f(a,ru J ) 

l<i<j<n k=0 



+ Kw) AkiD^r^E^ 1 ^ f(a, ru) 



b(o-,u)A (D i<T>u>) )E™ +1 f(o-,ruj) 

m j 

f £ I b(a,u)A k (D {aiU) )+ ^(<y^)X t3 A k ^ 1 (D { ^ ) )\r- k E^+ 1 - k f(<j,ru J ) 

k=l I \<i<j<n 

f ^ a y (cr,tj)Xyyl„ l (L> (CT ^ ) )r" m " 1 /(o-,ra;). 

l<i<j<n 
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It follows from the above expression that the assertion of the theorem holds for Q = E\P. Therefore, 
the proof is completed. □ 

5. Smoothness of / 

In this section, we will prove Propositions 13.11 and 13.21 

Proposition 5.1. There exist homogeneous polynomials Pk(cr,x) of degree k on G(p,n) (k = 
0, 1, 2, • • ■ ,) such that 



(-*) 



/M= E ^-Pk(<r,x) + (D^^ N )(a,cj;r), 

k=0 



where $jv(c,w;r)= / / 



e N (-i(y, ru))ip(ri, y) dydn, 



for N, (N — 1,2,3, ••• ,) and for (cr,x) € G(p,n) with x = ruj, ((a, to) £ F P ,n)- Here, as in 
Section^, ejv(i) denotes the Nth remainder term of the Taylor expansion of e . Moreover, as in 
\3.V^i . dij denotes the canonical and normalized measure on the set {ij 6 G q , n a C r\ c w -1 "}. 

Proof. By the definition of F, 

7(ct, ru>) = Dp q / TqV>{r\, ruj) dn 

= □£,/ ( e-^^( V ,y)dydr ] 



k\ 

k=0 



N-l 



E { -TT U tJ I <p(v,y)(y,rLo) k dydn 

+ (□<*> **)(a,a;;r). 

Since ip satisfies the moment condition (H'), there exist homogeneous polynomials Pfc(<7, x) of degree 
k on G(p, n) (k = 0, 1,2, ■ • • ), such that 

/ Pk(o-,ruj) da = / tp(r],y) (y,rw) k dy, for V(?7, rw) G G(<7, n) with rj ± uj. 

Applying the inversion formula for R£ to the both sides of the above equality, we have 



P k (a, ru) = / / ip(rj, y) (y, ru} k dydrj. 



Therefore, using these homogeneous polynomials P^, (k = 0, 1, 2, • • • , ), f{o~, ru) is written as 

w-i (_-\k 

k=0 

which completes the proof. □ 

Since G(p,n) \ (G pjl x {0}) = F P: „ x R + , the radial derivative E r also acts on a function on 
Fp^ n x R + . From now on, we extend the action of E r to F Pi „ x (R \ {0}) so that 

E r V(o-,Lj;r) = E r ^f(a,-u;;-r) for * e G°°(F P ,„ x (R \ {0})). 
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Proposition 5.2. For k, I 6 Z + with k + I = m, we have 

(r~ k E l r $ m+ i)(-,-;r) — ► 0, as r — > 0, in the topology o/C°°(Fp in ). 
Proof. The iV-th remainder term ejv(t) in the Taylor expansion of e* is expressed as 

_J_ 

(jv - 1)! y 



(5.1) e^(t) = t N g N (t), g N (t) = .„ 1 i . | / (1 - s) w e st 



Thus we have 



(5.2) 



$ m+ i(<r,£j;r) = / / (-i(y,ruj}) m+1 g m+1 (-i(y,ruj))(p(r],y)dydr] 



= '■' ' / / {-i(y,u)) m+1 g m+1 (-i(y,ruj))ip(Ti,y)dydri. 



Let us write 

(5.3) r" fc ^ {r m+1 (-i( J /,a;» m+1 flm+ iH(j/,M)}¥'('7.J/) = r^Wifa, y; w,r) 

Here we note that (£> m _|_i(?7,y;cJ,r) and its (higher order) derivatives with respect to y are bounded 
and integrable in y if |r| < 1. By l|5.2|) and l|5.3|l . we have 

(r~ k E l r $ m+1 )(u • cr,u • w;r) = r / / <p m+1 (ri,u ■ y;u,r) dydr], for u e O(n). 



Thus we have for Xi, • • • , Xr, G so(n) 
(5.4) 

(Xi---X L • (r- fc ^$ m+1 ))( ( j,t i ;;r) =r / / ((X x ) y ■ ■ • (X L ) y ■ </? m+1 )(77, y; w, r) dyefy. 



In equality (|5.4() . (Xj) y , (1 < j < L) acts on a function of y. Taking into account that y> G S(G(q, ri)) 
and y m+ i(— it) has bounded derivatives, we see easily that 

(5.5) sup |((X 1 ),-..(X L ) y • ^ m+ i)(77, y;u>,r)\ < C(l + \\y\\)-^ +1 \ foryG^. 

|r|<l 

In inequality 15.5|l . the constant C does not depent on (t],oj) G F q/n . As a result, for any L > 1 and 
for any Xi, • • ■ , X^ G so(n), there exists a constant C such that 

(5.6) sup \(X 1 ---X L -{r- k E l r <Z> m+1 ))(o-,u;r)\<C\r\, for r, (\r\ < 1), 

which proves the assertion. □ 

Lemma f3. 21 and Proposition 15 . 21 yield the following. 

Proposition 5.3. Let Q be an element ofu(so(n)) and let P = P^ m '(Ei, • ■ • , E n ) be a homogeneous 
element of lt(M n ) of degree m. Then, 

sup |(QP-D (p) $ m+ i)(a,w;r)| — 0, asr^O. 

(er,w)GF Pj „ 

Proo/. By Theorem Ol the differential operator P (m )(Pi, • • • , E n ) on P p ,„ x (M\ {0}) is written in 
the form 

m 

P^(E U ■ ■ ■ ,E n ) = ]T ^{D^r^E™-*, for (a,oj) G F p , n , r + 0, 

fc=0 

where (-D( CT is a differential operator on P Pjn of order at most k. Since Q G u(so(n)), Q can 
be regarded as a differential operator on F p , n . Moreover, acts on functions of (a, (J) G P Pi „ and 
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therefore commutes with the multiplication operator r and the radial derivative E r . Taking 
account of these two facts, we have 

m 

By Lemma \i. 21 is a continuous linear operator on C oc (F p _ n ) and so is Q o A i ^ n \D^ a uJ ^) o \J^ P \ 
Thus, by Proposition 15. 21 we have 

(QoA { ™\D {a ^)oa^ (r~ k E™- k <S> m+1 ) )(•, -;r) 0, as r -> 0, in the topology of C7°°(F p , n ). 
In particular, we have 

sup |(go4 m) (fl M )oDW (r- fc £r fc $ m+ i))^^r)h0, asr^O, 

(cr,w)eF p ,„ 

which completes the proof. □ 
Proof of Proposition I3TT1 

From now on, we will prove the smoothness of / around the set G Ptn x {0}. Since / is smooth 
on G(p,n) \ {G P:n x {0}), the proof is reduced to prove the following. 

Proposition 5.4. / is smooth near G p>n x {0}. 

Proof. It suffices to show that / is of class C m for any m (m = 0, 1, 2, • ■ ■ ,). We prove this by 
induction on rn. We have already proved that / is continuous on G(p, n), namely, / is of class C° 
on G(j>, n). In addition, we have already shown that / is smooth on G(p, n) \ (G Pj „ x {0}). 

We assume that / is of class C m on the set {(<j,x) £ G(p,n) \\\x\\ < 1}. Let us take any 
U G u(m(n)) of degree m + 1. From now on, we will prove that U ■ f(a, 0) exists and U ■ f(a, 0) is 
continuous at (a, 0) for any point (a, 0) £ G P! „ x {0}. 

Taking account of Proposition ^. II we may assume that U is written in the form (i) or (ii) below. 

(i) U = XQP, where X G so(n), Q £ u(so(n)) is of degree k and P £ u(R") is a homogeneous 
element of degree m — k. 

(ii) U — EiP, where P £ u(R") is a homogeneous element of degree m. 

So if we prove the following two lemmas, Lemma T5. II and Lemma f5. 21 then we see that / is of class 
G m+1 , and therefore, the proof of Proposition !.^ 41 is completed. □ 

Lemma 5.1. Let Q £ u(so(n)) be of degree k and let P £ u(R") be a homogeneous element of 
degree I. We assume that k + l = m. Let X £ so(n) and let a £ G p>n . Then, QP ■ f is differentiate 
at (er, 0) £ G Ptn x {0} with respect to X. Moreover, XQP ■ f is continuous at (a, 0). 

Proof. By the hypothesis of the induction, QPf is a continuous function on {(a, x) £ G(p, n)\\\x\ \ < 
1}. By Proposition 15. II 

f(a,ru)=^2 t^P j (o-,x) + (nW<f> l+1 )(o-,u;r), for x^rcu. 
4=0 3 ' 

Since Pj(a, x) is a homogeneous polynomial of deree j and P £ u(R") is a homogeneous differential 
operator of order I, P ■ Pj{cr,x) = if j < I — 1. Thus we have 

(5.7) QP-f(a,x) = ^(QP-P l )(a,x) + (QP-a^<f l+1 )(a,cj;r), {oxx = rcj. 
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Let r — ► in (| 5 . T|> . Then by Proposition 15 . 31 and the continuity of QP ■ /, we have 

QP-J(a,0) = ^-(QP-P l )(a,0). 

Hence, 

X(QP- f)(a, 0) = j t (QP ■ f)(e- tx a, 0)| t=0 

= f t (QP-Pl)(e- tX <r,0)\t=o 

= t± {X QP.P l )(a,Q). 

Note that by the definition of moment condition Pi 6 C°°(G(p, n)). Therefore, QP-f is differentiable 
at (a, 0) £ G p ,n x {0} with respect to X. 

Next, we will show the continuity of XQP ■ f at (a, 0). Similarly as in 1)5. 7fl . 

XQP J(a,x) = (XQP ■ Pi){a,x) + (XQP -D^ <P l+1 )(a,io;r), for a: = 

Again, by Proposition 15. 31 

lim (XQP-f)(a,x) = lim (XQP ■ P t )(a,x) + lim (XQP ■ n&> $j + i)(cr, w; r) 

= (XQP • P)(a, 0) = (XQP ■ f)(a, 0). 

□ 

Lemma 5.2. Let P G u(M n ) &e a homogeneous element of degree m. Then, P ■ f is differentiable 
at (cr,0) S G Pi „ x {0} with respect to E\. Moreover, E\P ■ f is continuous at (a, 0). 

Proof. If e! is parallel to a, obviously the assertion holds, so we assume that e x By the 

hypothesis of the induction, P ■/ is continuous on {(a, x) 6 G(p,n) | < 1}. Let u = Pr^ei 7^0. 
By the mean value theorem, 

(5.8) - t {P ■ f(a, -to) - P ■ f(a, 0)} = (£„P • f)(a, -6tv) = {EkP ■ f)(a, -Otv), 

for some 9, (0 < 6 < 1). 
By Proposition l5.ll 

m+l / .Nj 

(5.9) /(c7,ro;) = ^ iZ!L p. x ) + ( D W $ m+2 )( CT) w; r), for x = rc^. 

Since Pj(a,x) is a homogeneous polynomial of degree j and PiP £ u(K") is a homogeneous differ- 
ential operator of order m + 1, PiP • P,(cr, 2) = if j < m. Thus we have 

(5.10) (E x P-f)(c-,ru) = ^ 1 ,, (P 1 P-P m+1 )(a,x) + (P 1 FoD(p) $ m+2 )( (J , W ;r), for a: = rui. 

m + 1 ! 
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Hence by Proposition 15. HI 

(BiP ■ f)(a, -6tv) = > (E t P ■ P n+1 )(a, -9tv) 
(m + 1)1 

(5.11) + (^PoDW <Z> m+2 )(a,v/\\v\\; -9t\\v\ 

(_i)m+l 



(£iP-P m+ i)(<7,0), as i-»0. 



Therefore, it follows from (|5.8|) and (|5.11|) that P • / is differentiable at (a, 0) £ G p ,„ x {0} with 
respect to E\ and that 

(-i) m+1 

(5-12) PxP • f(a, 0) = ' (P • Pn+O^O). 

(m + lj! 

Next we will show that (E±P ■ /) is continuous at (a, 0). By (|5.10|l and Proposition EB 

{E t P ■ f){a, n>) = ' (E 1 P ■ P m+1 )(a, x) 
(m + lj! 

(5.13) +(P 1 Pon(f)$ m+2 )(a,^;r) 

(■\m+l 

— ► 7 — ~TTT ' Pm+i)(o, 0) = (PrP • f)(a, 0), asr^O, 

(m + lj! 

which proves the continuity of (PiP ■ /) at (a, 0). □ 
Finally we will prove that / is a Schwartz class function on G(p, n). 

Before going to the proof, let us recall from fGKl] and [El that a smooth function g on G(p, n) 
belongs to the Schwartz space S(G(p,n)) if for any nonnegative integers N and m and for any m 
vector fields Fi, • • • , Y OT e m(n) g satisfies 

sup \\x\\ N \(Y 1 ---Y m -g){a,x)\ < oo. 

Proof of Proposition UTT^l 

Proof. We start with the following. 

]{o,ruj) = D (p) / F q ip(r],ruj)dr]. 

Let us take any nonnegative integers m and iV. In addition, let us take any element Q G u(so(n)) 
and any homogeneous element P = P< m )(Pi, • ■ • , P„) e u(R n ) of de gree m. By Theorem 14.11 

r" (QP^(E lr -- ,E n )-f)(a,rLj) 

m - 

= r N Y, Q4 m} (D(^))r~ k E™- k D (p) / ?Mv, ™) dr, 

k=0 JaCvC^ 

171 

= £ QA^iD^D^ / rif-kErn-k Tqip{ ^ rLo) dl] 

m 

= Y,(Q° 4 m) UW ° S)( r N - k E™- k ^(-,r.) )(a, W ). 

fc=0 

where (P( CT>a ,)) is a differential operator on P p rl of order at most k and where S is the Radon 
transform from C°°(F q , n ) to G°°(P p ,„) defined by l|3.12[l . In the above computation, we used the 



(5.14) 
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fact that the multiplication operators r J (j = 1, 2, • • ■ ) commute with differential operators on F v . n 
and with the operator D^ p \ Since F q ip G S(G(q,n)), 

r N-k E rn-k r .) _^ Qj ag | r | ^ ^ 

in the topology of C°°(F q , n ). Moreover, in the summand of Q o Ajf (-D( CT , w) ) o □(?) o 5 is a 

continuous linear operator from C°°(Fg >n ) to C°°(F p , n ). Thus we have 

(Qo4™>(fl M )oDWoS)(rt;-'^(,r)) — > 0, as |r| - oo, 
in the topology of C°°(Fp tn ), from which we can conclude that 

(5.15) sup \r N (QP^iE!,--- ,E n )-f)(a,rio)\~^0, as \r\ oo. 

(o-,w)eF p ,„ 

Taking account of Proposition ^. II we see that (|5.15(l proves the assertion. □ 



6. The Support Theorem I 

Our objective in this section is to prove a support theorem for TZ^ p ' qS> based on the forward moment 
conditions (H'), generalizing Theorem 12.21 For this, we make use of the second-order differential 
operator A p on G(p, n) which acts as the Laplacian on each fiber a : 

A p /(cr,a;) = L a ±f(<r,x) 



A p is invariant under the motion group M(n) ( HI ). We define the differential operator A q on 
G(q, n) similarly. 

Using the operator A p , it makes sense to talk about harmonic polynomials on the fiber cr 1 - in 
G(p,n), as well as spherical harmonics on the unit sphere S a in cr • (These can also be obtained 
from harmonic polynomials on R"~ p by a translation by an appropriate element u G 0(n).) 

For any R > 0, define j3p to be the set of all p-planes (cr, x) at distance < R from the origin. We 
will use the following analogue of Theorem 12 . 31 for the partial Fourier transform T v . 

Theorem 6.1. The partial Fourier transform f i— > T v f maps T>((3p ) onto the space of all functions 
J- p f(a, Xlu) — F(a, u>; A) £ C°°(Fp in x R) satisfying the following conditions: 

(i) For each (cr, u>) G F p ^ n , the function A i— > F(a, lo; A) extends to a holomorphic function on C with 
the property that 



(6.1) 



sup 

(cr,w;A)eFp,„ xd 



(1 + \X\) N F{a,uj;X)e~^ ImXl 



< 00, 



for each N G Z+. 

(ii) For each k G Z + , /or each a G G p>n , and /or eac/i homogeneous degree k harmonic polynomial 
h a on a 1 - , the function 



F(a, u>; X)h a {oj) du> 



is even and holomorphic in C. 
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Proof. If / € £>(/3p ), then clearly the function A i-» F(cr, w; A) = J a± f(a,x)e- iX < x ' u ^ dx extends to 
a holomorphic function on C. If N G Z + , we have 

\\\ 2N F(a,u,;X) 



X 2N / f(*,x)e- iX < x '"Ux 
(-A^/Me^^efe 



< C max |(-Ap) JV /(<7,a;)|e ii|ImA| 

for all (er, w; A) E F Pi „ x C, which proves the estimate (|6.1[1 . Moreover, F satisfies condition (ii) 
above by Theorem 12 . HI applied to the Euclidean space a . 

Conversely, suppose T v f = F satisfies (i) and (ii). Let cr G G p ,„. Then - again by Theorem 12.31 
applied to the Euclidean space a 1 - - the function 



f(a,x) = (2tt)p- 



s„ Jo 



F(a, lu; A) e lX ^ x ^ A""^ 1 dX duo 



satisfies /(cr, x) = for all ||x|| > R. This shows that / is supported in (3^ . □ 

Our main result, the support theorem below, extends the classical support theorem, Theorem l2.2l 

Theorem 6.2. (Support Theorem for TZ^'^.) Assume that rank(G (p,n)) — rank(G(c/, n)). Sup- 
pose that f 6 5(G(p, n)) satisfies TZ^ p,q ^ f(f),y) = whenever \y\ > R. Then f is supported in 

Proof. Let (p = K^^f; then let S(r?,u;;A) = T q ip(r/, Xlu) as in (EH). From Theorem IO the 
function A t— > $(»7, w; A) extends to a holomorphic function on C for each (77, u) S Fj.„, and satisfies 
the estimate 



(6.2) 



sup 

(r),w;A)e_F g> „x([ 



(l + |A|) Ar $(77, W ;A)e- fi l ImA l 



< 00, for all iVeZ" 1 



Moreover, since X • = J- q (X • ip) for X S so(n), we have a similar estimate for X± ■ ■ ■ X m ■ $. 
Here X\ , • • • , X m G so(n) . Namely, for any nonnegative integer m and for any m vector fields 
Xi, ■ • ■ , X TO G so(n), we have 



(1 + \X\) N (X 1 ---X m - $)(,,, w; A) e-^l 



< 00, for all TV G Z+. 



(6.3) sup 

(l],u;A)ef,.„x( 

Let us fix an arbitrary nonnegative integer ./V and introduce a family of functions {H\ | A G C } in 
C°°{F q ^ n ) as follows. 

(6.4) H x (r,,u) = *fa,w;A)(l + |A|) w e -*l IroA l. 

Then the above two estimates (|6.2(l and H6.3(l show that {i?A | A G C } is a bounded set in C°°(F qn ). 
Now we put F(a, u>;r) — F p j '(a,ru>) as in (|3.19|) . Then by the (projection-slice) Theorem 13.41 



$(j7,w;r) 



F(a,w; r) da. 



(This is the same as equation (|3.11ll .) Just as with (|3.18() . for fixed u> the inversion formula (|3.10() 
may again be applied to recover F from <E>: 

F(a,u;r) = □£ 9 (S*)( £ r,«;r) = □<"> o S$(o-,u>;r). 
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This formula still holds when r is replaced by a complex parameter A. 
(6.5) F(a,uj;X) = nWoS^ujA). 

Next, we put G x (a,us) = F(a,u>; \)(1 + \\\) N e~ R \ ImX l Then G\ = o SH X . Since by Lemma 
IO and Lemma □<» o S is a continuous linear operator from C°°{F q , n ) to C°°(F p , n ), the set 
{Ga I A £ C} is bounded in C°°(F Pin ). In particular, we have 

sup |G A (cr,a;)|= sup \F(a, to; A)(l + |A|) Ar e - fl l ImA l | < +00, 

(a ; u;A)eF Pj „xC ((T,w;A)G-F p ,„xC 

which shows that the function A 1— » F(a,us; A) satisfies the uniform Paley- Wiener estimate 1)6. 

Next, we will prove that the function A i— > F(u,uj; A) is holomorphic in C for each (a,us) E F p , n - 
Let us take any closed curve P in C. Then by the continuity of o S, 



F{a,uj;\)d\ = S (J $ (v5 A ) dA ) = °- 



Here we used the fact that $ is holomorphic in A. Therefore, by Morera's theorem, A 1— ► F(a,co; A) 
is holomorphic. 

To finish the proof of Theorem 16.21 it remains to prove that for each homogeneous spherical 
harmonic h a of degree k on the unit sphere S a in cr -1 , the function 

(6.6) A i-» A~ fc / F(a,us;X)h a (u))dus 



is even and holomorphic in A. First, the fact that F(a,u>; A) = F(<r, —us, —A)) implies that (|6.6(l is 
even in A. 

In the calculations leading to Proposition l5.il which use the forward moment condition (H 1 ), the 
real parameter r can be replaced by the complex parameter A to give us the following expression. 

*(»7,w;A) = Xl^TT" / Pi(°,w)&r + / <ys(?7, y) (-%X(y, u;)) k ff fc (-*A(i/, w» dp, 

;=Q JcrCl) "'if 1 

where <j%(— iz) — ek{—iz)/{—iz) k is holomorphic in z and bounded if 2 is real. (See (|5.1() for an 
explicit expression of We write the last integral above as 

A fe / <p(v,y){-i{y,u)) k g k (-i\{y,u))dy = X k V k (ri,u; A), 

where ^^-(ry, u;; A) is a smooth function on F q>n x C and holomorphic in A for each (77, us) G Fg,n. 
By the inversion formula (|6.5|) (and (|3.1|1 for the function P; 6 C°°(G(p, n))), we get 

F(a,w;A) = ^t^P(<7,o/) + A fc □^5* fe (a, W ; A). 

Similarly as in the proof of holomorphicity of F(a, us;X), we can prove that □ ( p )S\t r fc(er, w; A) is 
holomorphic in A for each {a, us). Hence 

/ F(a : us;\)h a (us)dus = V / Pi(a,us) h a (us) dus + A fc / n^Sl^cx, w; A) h a (u) dus 

= X k [ n^S9 h (tr,ur,X)h a (u)du, 
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since Pi(<j, u>) is a degree I polynomial in lu G S ct , and so is a sum of spherical harmonics in S a of 
degree < I. The fact that the mapping 

o~, u>; A) h a (uj) dw 



is holomorphic in A now follows. The function J- p f(o~, Xlo) = F(a,ui; A) thus satisfies conditions (i) 
and (ii) in Theorem l6.ll and hence by that theorem / is supported in fi^. This completes the proof 
of Theorem IO □ 

7. Support Theorem II. 

In this section, we prove the support theorem for the Radon transform TZ(p>i*> : S(G(p,n)) — » 
S(G(q,n)) in the case when p < q and dim G(p,n) < dimG(g, n). 

Let O be a subset in R™. Throughout this section, we assume the following condition (A) on O. 

(A) For any £ G G(p, n) with £ C O c , there exists a hyperplane L such that £ C L C O c . 

The following support theorem holds. 

Theorem 7.1. Suppose that f G S{G(p,n)). If /(£) = /or a// £ C £> c ,. iften f{£) = /or 

a^Z^C O c . 

As a corollary, we have the usual support theorem for lZ( p ' q \ namely, 

Corollary 7.1. Let r > 0. Suppose ttai / G S(G(p,n)). IfK^ /(C) = /or a// ^, dist(^,0) > r, 
then f(£) = for all £, dM(£, 0) > r. 

Proof. If O — { x G R" | ||x|| < r }, then obviously O satisfies the condition (A). □ 

Remark 7.1. Similarly, if is a convex set, then satisfies the condition (A). So, in this case, the 
support theorem also holds. 

However, there are many cases when O satisfies the condition (A) but O is not necessarily convex. 
Even in such a case, the support theorem holds. 
The following are examples. 



Example 1. For a, 6 (a < b) and for r > 0, let 

01 = {x = (x 1 ,x 2 , ■ ■ ■ , x n ) G R" | (xi - a) 2 + x\ H Vx\< r 2 , x x < a }, 

2 = {x= (xi,x 2 ,-- • ,»n) G R" | (xi — 6) 2 + ^2 H ha; 2 < r 2 , xi > 6 }, 



and let O = 0\ U 2 ■ Then, O is no longer convex. But it is easily seen that O satisfies the condition 
(A). 

Example 2. For a, b (o < 6), let 

(7.2) Ox = {x= (x 1 ,x 2 ,--- ,x n ) e R™ | ii < a }, C 2 = {a = (si, x 2 , ■ ■ ■ ,x„) G K" | x x > b }, 

and let O = 0\ U 2 . Then, O c is a band domain. Similarly as in the above example, O is not 
convex, but O satisfies the condition (^4). 

Now, we proceed to prove Theorem 17. II The key is the injectivity of the Radon transform. 

The proof of Theorem 17.11 

Let us take an arbitrary p-plane £o which is included in O c . Then, by the condition (A), there 
exists a hyperplane L G G(n— 1, n) such that £ C L C O c . Let G(d, n; L) = { 7 G G(d, n) \ 7 C L }. 
Then the space of Schwartz class functions S(G(d, n; L)) on G(d, n; L) is defined in a similar manner 
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to S(G(d, n)). In addition, we can define a Radon transform lZ^' q ^ : S(G(p, n;L)) — > S(G(q, n; L)) 
as follows. 

(7.3) Kg< 9) f(t)= f f(l)dsl, for£eG(<z,ra;L), and for / e S(G(q, n; L)), 

where d^t is the canonical measure on the set {£ € G(p,n;L) £ C £ }. For / <E S(G(p,n)) in the 
statement of Theorem 17.11 let be the restriction of / onto the submanifold G(p,n;L). Then, 
Jl € S(G(p,n; L)). Moreover, by the definition of TZ^'^ and by the assumption of the Theorem, 
we have 

(7.4) ni M) f L (0=n^f(0 = Q, io^eG(q,n;L). 

In fact, if £ e G(q, n; L), then £ C O c and therefore /(£) = 0. 

By applying a suitable translation and a suitable orthogonal transformation to L, we may assume 
that L = Kei © • • ■ ©Ke„_i(= K n_1 ). Then, the Radon transform TZ^' q) is nothing but the Radon 
transform from S{G{p,n — 1)) to S(G(q,n — 1)) associated with inclusion incidense relation. By 
the assumption that p < q and dim G(p,n) < dimG(q, n), we see easily that dim G(p,n — 1) < 
dim G{q 1 n— 1). So, by Theorem 6.4 and Remark 3 in Section 6 of our previous paper |GK1| . TZ^' q ^ 
is injective. Hence, by (|7.4I) . /l(^) = for £ 6 G(p 7 n;L). In particular, /(^o) = /l(-^o) = 0, which 
completes the proof. □ 

Finally, as an application of the above support theorem, we give a range characterization ofTZ^ p ' q ^ 
in the category of compactly supported smooth functions. 

Let C£°(G(d,n)) denote the space of compactly supported smooth functions on G(d,n). Then, 
it is easily seen that since p < q the Radon transform TZ^P'i) maps C^°(G(p, n)) to C^°(G(q, n)). As 
we stated in the introduction, since dim G(p,n) < dim G(q,n) the image of TZ^ p ' q ^ is characterized 
as the solution space of an M(n)-invariant differential equation of order 2p + 4 of the form, 

(7.5) dv(Q 2 p+A)v = 0. 

Here Q 2p +4 is an element in 3(m(n)) and is expressed as the sum of the squares of "PfafHans" of 
order p+2. (See Gonzalez and Kakehi [GK1| for the definition of the operator Q 2p +A and its detailed 
properties.) 

Since C%°(G(d,n)) C S(G(p 7 n)), the support theorem (Theorem 17. 1[) and the range theorem for 
ftM (Theorem 7.7 of [SET]) yield the following. 

Theorem 7.2. Suppose that p < q and dimG(p,n) < dimG(q,n). A function ip G C£°(G(q,ri)) 
belongs to the range TZ^ p ' q \C^°(G(p,n))) if and only if dv{Q2 P +A} (p = 0. 
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